Medical viewing system and method for detecting and enhancing structures in noisy images

ABSTRACT

A medical viewing system for displaying a sequence of images of a medical intervention that comprises moving and/or positioning a tool in a body organ, which tool is carried by a support to which at least one marker is attached at a predetermined location with respect to the tool, comprising means for acquiring the sequence of images, and for processing said images during the medical intervention, wherein: extracting means for automatically extracting at least one marker that is attached to the tool support and that neither belongs to the tool nor to the body organ, and yielding the marker location information; computing means for automatically deriving the tool location information from the marker location information, and enhancing means for improving the visibility of the tool and/or the body organ in order to check whether the medical intervention stages are successfully carried out.

FIELD OF THE INVENTION

The invention relates to a medical viewing system for displaying asequence of images of a medical intervention that comprises movingand/or positioning a tool in a body organ. The invention also relates toa computer executable image processing method to be used in said system.The invention further relates to a medical examination apparatus coupledto such a system. The invention finds for example its application in themedical field of cardiology, for extracting, registering and enhancingthin objects of interest such as stents and vessel walls inarteriograms.

BACKGROUND OF THE INVENTION

A method for extracting stents in medical images is already known fromthe publication entitled “Deformable Boundary Detection of Stents inAngiographic Images”, by Ioannis Kompatsiaris et alii, in IEEETRANSACTIONS ON MEDICAL IMAGING, VOL.19, No. 6, JUNE 2000, pages652-662. This document describes an image processing method fordeformable boundary detection of medical tools, called stents, inangiographic images. A stent is a surgical stainless steel coil that isplaced in the artery in order to improve blood circulation in regionswhere a stenosis has appeared. Assuming initially a set ofthree-dimensional (3-D) models of stents and using perspectiveprojection of various deformations of the 3-D model of the stent, alarge set of two-dimensional (2-D) images of stents is constructed.These synthetic images are then used as a training set for deriving amulti-variate density estimate based on eigenspace decomposition andformulating a maximum-likelihood estimation framework in order to reachan initial rough estimate for automatic object recognition. Thesilhouette of the detected stent is then refined by using a 2-D activecontour (snake) algorithm, integrated with an iterative initializationtechnique, which takes into consideration the geometry of the stent.

As disclosed in the cited publication, when a narrowing called stenosisis identified in a coronary artery of a patient, a procedure calledangioplasty may be prescribed to improve blood flow to the heart muscleby opening the blockage. In recent years, angioplasty increasinglyemploys a stent implantation technique. This stent implantationtechnique includes an operation of stent placement at the location ofthe detected stenosis in order to efficiently hold open the diseasedvessel, as illustrated by FIG. 2 of the cited publication. Stentplacement helps many patients to avoid emergency heart bypass surgeryand/or heart attack (myocardial infraction). The stent, as illustratedby FIG. 1 of the cited publication, is a small, slotted, stainless steeltube cut by a precision laser for forming a coil. It is wrapped tightlyaround a balloon attached to a monorail introduced by way of a catheterand a guide-wire forming a device called balloon-tipped catheter. Thisballoon-tipped catheter is introduced into the artery through a smallincision. Once in place, the balloon is inflated in order to expand thecoil. Once expanded, the stent, which can be considered as a permanentimplant, acts like a scaffold keeping the artery wall open. This allowsmore blood flow to the heart muscle.

SUMMARY OF THE INVENTION

The stent, the monorail and the thin guide-wire are observed in noisyfluoroscopic images. They show low radiographic contrast that makesevaluation of the placement and expansion of said stents at an accuratelocation very difficult. Also, during the operation of stentimplantation, the monorail, with the balloon and stent wrapped aroundit, is moving with respect to the artery, the artery is moving under theinfluence of the cardiac pulses, and said artery is seen on a backgroundthat is moving under the influence of the patient's breathing. Thesemovements make the following of stent implantation under fluoroscopicimaging still more difficult to visualize. In particular, thesemovements make zooming inefficient because the object of interest mayget out of the zoomed image frame. Clinical problems are associated withinadequate expansion of the stent, improper placement of the stent, andgap or overlap between several stents. Studies revealed that more thaneighty per cent of stents might be insufficiently dilated despite anapparently angiographically successful deployment. Inadequately expandedstents can locally disrupt blood flow and cause thrombosis.

The method that is disclosed in the cited publication deeply relies onthe identification of the stent in the angiographic images. This knownmethod has steps of forming sets of 3-D models of stents, steps ofconstructing sets of 2-D images from the 3-D models and steps ofmatching the 2-D models to the 2-D images of the stent in thecardiograms. This method would present a calculation load that isactually too heavy for real time processing of a sequence of imagesneeded in the intervention phase of stent implantation. Also, thepractitioners are more and more demanding about the resolution of theimages. So, the proposed method may be preferably only used in apost-intervention phase.

Instead, it is an object of the invention to propose a medical viewingsystem that has means to process medical images in order to be usedduring the intervention phase. For visualizing the intervention of stentimplantation, this system has means to solve the problems ofautomatically extracting features that permits of accurately positioninga balloon with respect to the stenosed zone of the artery, for examplefor inflating the balloon so as to expand the lumen of the artery,instead of, or before expanding a stent; and automatically extractingfeatures that permit of accurately positioning a balloon, with a stentwrapped around it, for expanding the stent.

According to the invention, these problems are not solved by merelyextracting the stent or the artery walls. Instead, these problems aresolved by extracting features that do not belong to the objects to beactually finally of interest for the practitioner such as stent orartery walls. As a matter of fact, as explained above, said objects areby nature badly contrasted, represented on an already noisy backgroundand submitted to motions. According to the invention, a medical viewingsystem is provided comprising means for acquiring a sequence of images,and for processing and displaying said images during the medicalintervention, wherein means for automatically extracting at least onemarker that is attached to the tool support and that neither belongs tothe tool nor to the body organ, and for yielding the marker locationinformation, means for automatically deriving the tool locationinformation from the marker location information, and means forimproving the visibility of the tool and/or the body organ in order tocheck whether the medical intervention stages are successfully carriedout. It is also an object of the invention to provide such a systemwhose image processing means is user-actuatable. It is a further objectof the invention to propose such a viewing system, wherein theprocessing means comprises registering means, enhancing means andzooming means that permit of accurately visualizing the artery walls atthe stenosis location; and/or checking stent deployment with respect tothe artery walls in regions of interest.

Such a system is claimed in Claim 1 and in dependent Claims. An imageprocessing method to be used in the system, a program product toimplement the steps of the method of the invention and an examinationapparatus for helping visualization of interventions having such asystem are further claimed.

In an application to angioplasty, the system comprises means to extractfeatures not belonging to the stent or to the artery, among which afeature called guide-wire tip located at the extremity of the guide-wireguiding the monorail; and/or at least one feature called balloon-markerlocated on the monorail at a given position with respect to the balloon;preferably there are two balloon markers disposed at each extremity ofthe balloon. The guide-wire tip belongs neither to the artery walls norto the stent, since it belongs to the guide-wire. Also, theballoon-markers belong neither to the vessel walls nor to the stentsince they belong to said monorail. Using the method of the invention,the guide-wire tip and the balloon-markers are accurately detected andthis detection further permits of accurately localizing the stenosedartery walls and the stent. Advantages of the system and method of theinvention are that they improve the results of the intervention phase ofstent implantation in a way that is precise and robust; they provideaccurate stenosis detection and stent deployment checking during theintervention phase or in a post-intervention phase.

LIST OF DRAWINGS

Embodiments of the invention are described hereafter in detail inreference to diagrammatic figures wherein:

FIG. 1 is a functional block diagram of means of the system;

FIG. 2 illustrates the device constituted by a catheter, a guide-wireand a monorail with a balloon and a stent wrapped around it;

FIG. 3A to FIG. 3F illustrate the information obtained during theintervention stages of angioplasty using the system of the invention;

FIG. 4 illustrates the step of marker registration;

FIG. 5 is a functional block diagram of a medical examination apparatususing said system.

DESCRIPTION OF EMBODIMENTS

The invention relates to a viewing system, and to a computer executableimage processing method that is used in the viewing system, fordetecting, localizing, registering, enhancing and zooming structures innoisy images. The viewing system and the image processing method of theinvention are described hereafter in an example of application to themedical field of cardiology. In said application, the objects ofinterest are organs such as arteries and tools such as balloons orstents. They are observed during a medical intervention calledangioplasty, in a sequence of X-ray fluoroscopic images calledangiograms. The system and method may be applied to other tools thanstents and balloon in other intervention than angioplasty. For instance,the objects of interest may be electrodes and the organ may be thebrain.

The stent implantation is a medical intervention that usually comprisesseveral stages for enlarging an artery at the location of a lesioncalled stenosis. In a preliminary stage, the practitioner localizes astenosis 40 a in a patient's artery 41 as best as possible in medicalimages. This medical intervention includes stages of:

-   a) Referring to FIG. 3A, introduction in the artery 41, using a    catheter 9, of a thin guide-wire 1 that extends beyond the extremity    of the catheter 9, and passes through the small lumen of the artery    portion 40 a at the location of the stenosis.-   b) Referring to FIG. 3B, introduction of a monorail 10, which is    guided by said guide-wire 1 that passes through the opening 3 of the    monorail 10 as illustrated by FIG. 2, and which has a first balloon    14 a wrapped around its extremity, without stent; and positioning    said first balloon 14 a in the artery portion 40 a at the stenosis    location.-   c) Referring to FIG. 3C, inflation of this first balloon 14 a, which    becomes the inflated balloon 14 b, for expanding the narrow lumen 40    a of the artery 41 at the location of the stenosis to become the    enlarged portion 40 b of the artery; then, removal of the first    balloon 14 b with the first monorail 10.-   d) Referring to FIG. 3D, again using the catheter 9 and the thin    guide-wire 1, introduction of a second monorail 20 with a second    balloon 24 a wrapped around its extremity, and with a stent 25 a    around said second balloon 24 a; and positioning said second balloon    with the stent at the location of the stenosis in the previously    expanded lumen 40 b of the artery 41.-   e) Referring to FIG. 3E, inflation of the second balloon 24 a to    become the inflated balloon 24 b in order to expand the coil forming    the stent 25 a, which becomes the expanded stent 25 b embedded in    the artery wall.-   f) Referring to FIG. 3F, considering the expanded stent 25 b as a    permanent implant, removing the second balloon 24 b, the second    monorail 20, the guide-wire 1 and catheter 9.

The medical intervention called angioplasty is difficult to carry outdue to badly contrasted medical images, where the guide-wire, balloon,stent and vessel walls are hardly distinguishable on a noisy backgroundand are moreover submitted to motions. Some phases of this interventionare very critical, so checking actions have to be performed during thesephases. These phases are:

-   Phase 1) after stage a) it is important to check the position of the    guide-wire 1 with respect to an artery portion of interest 40 a,    such as the stenosed zone.-   Phase 2) between stages b) and c), it is important to check whether    the first balloon 14 a is placed correctly in coincidence with the    stenosed zone 40 a of the artery 41.-   Phase 3) after stage c), it is important to inspect the artery    portion in order to verify the removal of the stenosis by the    previous expansion of the first balloon.-   Phase 4) between stages d) and e), it is critical to check whether    the stent 25 a is accurately positioned with respect to the expanded    lumen 40 b of the artery 41.-   Phase 5) After stage e) it is very important to check whether the    balloon 24 b is sufficiently inflated, and whether the stent 25 b is    successfully expanded in the artery to be embedded in the artery    wall, in order to gauge the result of the operation.-   Phase 6) after stage f), it is important to be able to perform    checking of the condition of the stent 25 b after implantation in    the artery portion.

The system of the invention has means to perform these actions duringthe intervention at the above-cited critical phases. The user is theactor of the medical intervention and can have the possibility tointervene at each phase. First of all, the user might choose a region ofinterest in the images. Besides, the user has at his disposal controlmeans 58, shown in FIG. 5, to activate and control the image processingmeans. These control means comprises starting means and stopping meansfor the user to start the processing operation, to control the durationof the processing operation and to end the processing operation. Theprocessing means are actuated by the user during one of the. above-citedphases, for example while not moving the tool or tools.

Referring to FIG. 1, these image processing means include extractingmeans 101 that solves the problems of automatically and accuratelylocalizing specific features in the sequence. Once the specific featurelocation 201 has been determined by the automatic extracting means 101,registering means 102 provides registered sequence images 202, based onthe specific feature location 201. Then, enhancing means 103 yieldsimages 203 with enhanced objects of interest, such as the tool and/orthe artery walls. Zooming means 104, that are now applied to stableobjects of interest in the registered images, permits an improvedvisualization of the regions of interest with the tools and the bloodvessels in processed images 204.

The processed images permit of checking the position of the guide-wirewith respect to the stenosed zone of the artery; of checking theposition of the first balloon with respect to the stenosed zone beforeexpanding the lumen of the artery; permits of checking the position ofthe second balloon, with the stent wrapped around it, before stentexpansion and permits of finally checking the expanded stent.

According to the invention, these specific features are not merely thestent or the artery walls. Instead, these specific features do notbelong to the badly contrasted stent or vessel walls, which are theobjects that are actually finally of interest for the practitioner.According to the invention, the system has extracting means 101 toautomatically and accurately extract specific features among which afeature called guide-wire tip 2 located at the extremity of the thinguide-wire 1 guiding the monorail 10 or 20; and/or at least one featurecalled balloon-marker 11, 12 or 21, 22 located respectively on themonorail 10, 20 at a given position with respect to the balloonrespectively 14 a, 24 a; preferably there are two balloon-markersdisposed at each extremity of the balloon. The guide-wire tip 2 belongsneither to the artery walls 41 nor to the stent 25 a, since it belongsto the guide-wire 1. Also, the balloon-markers 11, 12 or 21, 22 belongneither to the vessel walls 41 nor to the stent 25 a since they belongto the monorail 10 or 20. These markers have a specific easilyrecognizable shape, and are made of a material highly contrasted in theimages. Hence they are easy to extract. The system processing means 102,103, 104 permit of accurately deriving the location of the balloons 14a, 24 a, 14 b, 24 b, since the balloons have specific locations withrespect to the balloon-markers. Also, the stents 25 a, 25 b areaccurately localized, since the stents have specific locations withrespect to the balloon-markers though said stents are not attached tosaid balloon-markers. The automatic location of said markers permits ofperforming the actions required at phases 1), 2), and 4). Moreover, thesystem processing means permit of automatically and accurately detectspecific features such as boundaries of the previously localized arterywalls and stent. The result of these image processing steps furtherpermits of checking the proper expansion of the lumen of the arteryafter the inflation of the first balloon; and permits of checking theproper expansion of the stent after the inflation of the second balloon.Hence, the result of these image processing operations permits ofperforming the actions required at phases 3), 5), and 6).

Referring to FIG. 4, for instance two markers A_(Ref), B_(Ref) have beendetected in an image of the sequence, called image of reference, whichmay be the image at starting time. The markers A_(Ref), B_(Ref) may beselected by automatic means. They are extracted using extracting means101 of the system. Then, registering means 102 of the system, using themarker location information A_(Ref), B_(Ref) in the reference image andcorresponding extracted markers A′_(t), B′_(t) in a current image of thesequence, are operated for automatically registering the current imageon the reference image. This operation is performed by matching themarkers of the current image to the corresponding markers of thereference image, comprising possible geometrical operations including: Atranslation T to match a centroid C_(t) of the segment A′_(t) B′_(t) ofthe current image with a centroid C_(Ref) of the segment A_(Ref) B_(Ref)of the reference image; a rotation R to match the direction of thesegment A′_(t) B′_(t) of the current image with the direction of thesegment A_(Ref) B_(Ref) of the reference image, resulting in a segmentA″_(t) B″_(t); and a dilation A for matching the length of the resultingsegment A″_(t) B″_(t) with the length of the segment A_(Ref) B_(Ref) ofthe reference image, resulting in the registered segment A_(t) B_(t) forthe registered current image. Such operations of translation T, rotationR and dilation Δ are defined between the images of the sequence and thereference image, resulting in the registration of the whole sequence.This operation of registration is not necessarily performed on all thepoints of the images. Zones of interest comprising the markers may bedelimited. This operation of registration permits of minimizing theeffect of respective movements of the objects of interest, such asvessels, guide-wire, balloons and stent, with respect to a predeterminedimage referential. In the registered images, the user can easily performzooming Z on objects of interest. Preferably, two markers, or more, areused for better registration.

In the registered sequence, an object of interest, such as stent, can beenhanced by the enhancing means 103. For this operation, the shape anddimension of the stent is a-priori knowledge that can be stored inmemory means of the system. The boundaries of the stent aresubstantially parallel to the segment formed by the balloon-markers, andat a distance from this segment that can be derived from the a-prioriknowledge. So, these boundaries are detected, extracted and enhanced bythe enhancing means 103 of the system. Also, the artery walls can bedetected, extracted and enhanced by the enhancing means 103 in a similarmanner.

The registered images are preferably filtered for minimizing noise. Thesystem enhancing means 103 may comprise noise filter means. In anexample, the registered images are integrated by averaging means appliedto the intensity of the points. By this operation, the details ofobjects, such as vessels, which are in time concordance, are enhancedwhile the details of the background, which are not in time concordance,are minimized. The registered images are also preferably submitted tospatial background subtraction means of the enhancing means 103.Background subtraction means permits of eliminating large contrastedzones and permits of again enhancing the objects of interest.

A computer executable image processing method to be used in a system asabove described has steps of processing a sequence of digital imagesduring the medical intervention, comprising automatically extracting atleast a marker that belongs neither to the tool nor to the body organ,automatically determining the marker location information in the imagesof the sequence, deriving the tool location information from the markerlocation information, and processing the images to improve thevisibility of the tool and/or of the organ. The method permits ofdisplaying the images during the medical intervention for the user toposition the tool in the organ at a specific location using the markerlocation information.

FIG. 5 shows a diagram of a medical examination apparatus 50. Theapparatus has means 51 for acquiring digital image data of a sequence ofimages, and is coupled to a medical viewing system 53 as describedabove, for processing these data according to the processing methodcited above. The medical viewing system is generally used in theintervention room or near the intervention room for processing real timeimages. Steps of the present method can be applied on stored medicalimages, for example for estimating medical parameters. The system forprocessing the data of the stored images is then called medical viewingstation. The medical examination apparatus provides the image data byconnection 57 to the system 53. The system provides processed image datato display means and/or storage means. The display means 54 may be ascreen. The storage means may be a memory MEM of the system 53. Saidstorage means may be alternately external storage means. This imageviewing system 53 may comprise a suitably programmed computer, or aspecial purpose processor having circuit means such as LUTs, Memories,Filters, Logic Operators, that are arranged to perform the functions ofthe method steps according to the invention. The system 53 may alsocomprise a keyboard 55 and a mouse 56. Icones may be provided on thescreen to be activated by mouse-clicks, or special pushbuttons may beprovided on the system, to constitute control means 58 for the user tostart, to control the duration or to stop the processing means of thesystem at chosen stages or phases.

1. A medical viewing system for displaying a sequence of images of a medical intervention that comprises positioning a tool in a body organ, the tool configured to be carried by a support, wherein the support includes at least one localizing feature marker at a predetermined location with respect to the tool, comprising: acquisition means for acquiring the sequence of images; processing means for processing said images of the sequence of images; extraction means for extracting information from said images related to the at least one localizing feature marker of the support and for use in computing marker location information; computer means for calculating tool location information as a function of the marker location information; and enhancement means for enhancing a visibility of the tool and the body organ within the displayed sequence of images as a function of the calculated tool location information.
 2. The system of claim 1, further comprising controller means to direct the processing means in connection with a selected stage of the medical intervention.
 3. The system of claim 1, wherein the computer means further comprises: selection means for selecting a reference image of the sequence and at least one reference marker in the reference image; and registration means configured to use marker location information extracted from (i) the reference image and (ii) a current image of the sequence, for registering the current image of the sequence on the reference image by matching the extracted marker of the current image to the reference marker of the reference image.
 4. The system of claim 1, wherein the enhancement means further comprises: storage means for storing information related to dimensions of the tool or the body organ; and computer means, configured to use the marker location information and the information related to dimensions of the tool or the body organ, to extract and enhance image features of the tool or the body organ within the displayed sequence of images.
 5. The system of claim 4, wherein the enhancement means for enhancing further comprises: zoom means for zooming on a region of interest within the displayed sequence of images; noise filter means for filtering noise as part of enhancing objects of interest within the displayed sequence of images; and background subtraction means for subtracting background from the displayed sequence of images.
 6. The medical viewing system of claim 1, wherein the tool includes a balloon carried by a monorail support, said support including two attached localizing balloon markers; the extraction means determines balloon marker location information; the computer means derives balloon location information at least in part from the balloon marker location information and the enhancement means improves the visibility of the balloon or the area in the vicinity of the balloon; and further comprising: a display means for displaying the images during the intervention.
 7. The system of claim 6, further comprising a guide wire for guiding the monorail, said guide wire having a tip including a tip marker for localizing; and wherein the extraction means further determines tip marker location information.
 8. The system of claim 7, wherein the tool further includes a stent wrapped around the balloon; and the computer means calculates stent location information based at least in part on the balloon marker location information.
 9. The system of claim 6, further comprising: activation means for the user to activate or stop, at a selected stage of the medical intervention, the processing means applied to the sequence of images for improving the visibility of the tool or the body organ, said selected stage including at least one stage chosen from among a group of stages consisting of positioning the guide wire tip with respect to an artery portion; positioning the balloon markers with respect to the location of the portion of the artery, before a stage of balloon inflation or a stage of stent deployment; and inspection of the artery portion after the stage of balloon inflation or the stage of stent deployment.
 10. The system of claim 6, wherein the computer means further comprises: selection means for selecting a reference image and location means for calculating information related to locations of the balloon markers in the reference image; and registration means for using the balloon marker location information in the reference image and in other images of the sequence for registering the images of the sequence by matching corresponding balloon markers of a current image to balloon markers of the reference image.
 11. The system of claim 10, wherein the enhancement means further comprises: storage means for storing previously gathered information related to at least one item selected from the group consisting of the balloon, a stent associated with the balloon, an area surrounding the balloon, and an area surrounding the stent, wherein said computer means uses the balloon marker location information and the previously gathered information for localizing the at least one item, and wherein said enhancing means enhances the visibility of the at least one item within the displayed images as a function of the localizing of the at least one item.
 12. An image processing method for visualizing a medical intervention that includes positioning a tool in a body organ, comprising the steps of: acquiring a sequence of images during the medical intervention; and processing said images during the medical intervention, wherein the processing includes extracting from each image a marker, wherein the marker belongs neither to the tool nor to the body organ, computing marker location information of respective markers in the images of the sequence, computing tool location information from the marker location information, and enhancing the images as a function of the computed tool location information to improve the visibility of the tool within the enhanced images; and displaying the enhanced images during the medical intervention.
 13. The method of claim 12, wherein the processing step further comprises at least one step selected from the group consisting of registering the images of the sequence on a reference image; filtering noise in the images; subtracting background from the images; and zooming objects of interest in the images.
 14. A computer-readable medium containing machine-readable instructions executable by a computer for carrying out the method of claim
 12. 15. A computer-readable medium containing machine-readable instructions executable by a computer for carrying out the method of claim
 13. 16. The method of claim 12, wherein the images of the sequence comprise angiograms. 